Application of information theory and statistical learning to anomaly detection

In today\u27s highly networked world, computer intrusions and other attacks area constant threat. The detection of such attacks, especially attacks that are new or previously unknown, is important to secure networks and computers. A major focus of current research efforts in this area is on anomaly detection.;In this dissertation, we explore applications of information theory and statistical learning to anomaly detection. Specifically, we look at two difficult detection problems in network and system security, (1) detecting covert channels, and (2) determining if a user is a human or bot. We link both of these problems to entropy, a measure of randomness information content, or complexity, a concept that is central to information theory. The behavior of bots is low in entropy when tasks are rigidly repeated or high in entropy when behavior is pseudo-random. In contrast, human behavior is complex and medium in entropy. Similarly, covert channels either create regularity, resulting in low entropy, or encode extra information, resulting in high entropy. Meanwhile, legitimate traffic is characterized by complex interdependencies and moderate entropy. In addition, we utilize statistical learning algorithms, Bayesian learning, neural networks, and maximum likelihood estimation, in both modeling and detecting of covert channels and bots.;Our results using entropy and statistical learning techniques are excellent. By using entropy to detect covert channels, we detected three different covert timing channels that were not detected by previous detection methods. Then, using entropy and Bayesian learning to detect chat bots, we detected 100% of chat bots with a false positive rate of only 0.05% in over 1400 hours of chat traces. Lastly, using neural networks and the idea of human observational proofs to detect game bots, we detected 99.8% of game bots with no false positives in 95 hours of traces. Our work shows that a combination of entropy measures and statistical learning algorithms is a powerful and highly effective tool for anomaly detection

Battle of Botcraft: Fighting Bots in Online Games with Human Observational Proofs

The abuse of online games by automated programs, known as game bots, for gaining unfair advantages has plagued millions of participating players with escalating severity in recent years. The current methods for distinguishing bots and humans are based on human interactive proofs (HIPs), such as CAPTCHAs. However, HIP-based approaches have inherent drawbacks. In particular, they are too obtrusive to be tolerated by human players in a gaming context. In this paper, we propose a non-interactive approach based on human observational proofs (HOPs) for continuous game bot detection. HOPs differentiate bots from human players by passively monitoring input actions that are difficult for current bots to perform in a human-like manner. We collect a series of user-input traces in one of the most popular online games, World of Warcraft. Based on the traces, we characterize the game playing behaviors of bots and humans. Then, we develop a HOP-based game bot defense system that analyzes user-input actions with a cascade-correlation neural network to distinguish bots from humans. The HOP system is effective in capturing current game bots, which raises the bar against game exploits and forces a determined adversary to build more complicated game bots for detection evasion in the future